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ABSTRACT 

Users of optical data disk (CD-ROM) materials at the 
Main Library workstations of the State University of New York at 
Albany were surveyed to examine the problems th*y encountered while 
conducting their searches, identify factors that interfere wit u their 
task performance, determine their needs with regard to improvement in 
the databases they used, and evaluate the CD-ROM search service 
provided by the reference department. Data were gathered using a 
12-item questionnaire completed by 200 non-repeat users (4 faculty 
members, 104 graduate students, 1 staff member, and 91 
undergraduates) affiliated with the university. Results indicate that 
patrons made little use of workshops that the university had provided 
to introduce CD-ROM, and that they tended to request assistance 
rather than use the database search guides at the workstations. 
Graduate students wore inclined to make better preparation for their 
searches than were undergraduates, but only a few patrons used 
Boolean logic to formulate their search statements. The responses and 
comments from searchers provide valuable information that library 
personnel can use to improve service. Twenty-one tables present study 
data, and an appendix lists some er suggestions. (Contains 12 
references.) (SLD) 
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Abstract 



This paper describes a study that surveyed CD-ROM users at the Main Library 
workstations of the St ate University of New York at Albany to examine the problems 
they encountered while conducting their searches, identify the factors that interfere with 
their task performances, determine users' needs with regard to the improvement in the 
databases they used and the CD-ROM search service provided by the Reference 
Department. Data was gathered from 200 non-repeat SUN Y/ Albany affiliated users 
through a 12-item questionnaire. (A copy of the questionnaire is available from the 
author on request.) 



The growth in the development of CD-ROM databases as a new and important 
access medium for bibliographic reference sources has been tremendous during the past few 
years. With the proliferation and increased availability of CD-ROMs, indexes and other 
information tools in CD-ROM format have become a favorite in library literature and are 
prevalently used in the academic and research libraries in the United States. CD-ROM 
producers always claim that their products are easy to use. From my experience, working 
at the Electronic Reference Station at the New York State Library, I have come to realize 
that no matter how user-friendly they are, there are still problems that users come across 
while they conduct their searches. In order to determine the potential problems, a survey 
was designed to investigate the difficulties the CD-ROM users encounter at the Main 
Library workstations of The University at Albany, State University of New York, and to 
identify the factors that interfere with their task performances. This data will enable 
librarians to find ways to improve the CD-ROM database searching services. Instead of 
measuring user satisfaction with CD-ROM products as most of the previous studies have 
done, this project focused on the problems identified by the users and their needs as well. 

Many recent articles have investigated CD-ROM applications in academic libraries. 
Most of them concentrated on the responses of patrons to a variety of CD-ROM products 
and services in general. For instance, "Users' Reactions to CD-ROM: The Penn State 
Experience" presented by Cindy Faries 1 provided information about the user community, 
the most frequently used databases, users' perception of the help available to them, their 
desired type of instruction and suggestions for improving the services. 

The study conducted by Ramona J. Steffey and Nikki Meyer 2 in the three libraries 
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of Vanderbilt University evaluated user success and satisfaction with various CD-ROM 
databases. The criteria used .n their evaluation are ease of use, satisfaction with the 
number and quality of retrieved citations and the amount of time spent on searching. 

Patricia Lynn and Karen Bacsanyi's 3 research reported on the development practice 
and success of five CD-ROM instructional methods explored at the Wayne State University 
Library. Survey results of patrons' using PsycLit, ERIC, Compact Disclosure and Social 
Sciences Index at the Purdy/Kresge Library were also included. 

Two surveys, one designed for patrons, the other for faculty, undertaken in the 
summer of 1988 by Jean Reese 4 at the Vanderbilt University Libraries, focused on the 
impact of CD-ROM technology on library staff as well as on students and faculty members. 
Different aspects of impacts were described, such as increased workload for reference 
librarians, user training, furnishing the instruction aids, the increased requests of interlibrary 
loan and the drop-off of online searching. 

The study conducted by Howard Silver 5 between January and March 1987 examined 
public response to MEDLINE, PsycLit, and Science Citation Index. In addition, topics 
related to the CD-ROM database service at the Hahnemann University Library were 
discussed. They include the instruction of hardware and software, staff training, changes in 
workload, access and promotion of the service, and bibliographic instruction. 

Other articles examined experiences patrons had in using simply one product. The 
following are some of the examples. 

Jo McClamroch, Linda Lawrence Stein, and Edgar Williamson 6 designed a 
questionnaire to elicit opinions from end-users about MLA on CD-ROM. Survey results 
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indicated that this product is easy to use for both the novice and the experienced searcher. 
Patron satisfaction was reflected by their positive comments about the system's currency, 
speed, printing and searching capabilities. The drawbacks concerning the respondents were 
lack of adequate printed documentation and on-screen instruction. 

In Spring 1989, Somporn Puttapithakporn 7 identified and categorized problems 
novices users encountered in a database searching task with the Version 1.4 of ERIC on 
SilverPlatter. Thirty-three students enrolled in Introduction to Information Science at 
Indiana University-Bloomington participated in this study. The article served two p arposes. 
One was to propose a taxonomy for analyzing user problems and errors as an evaluation 
tool to the performance of a CD-ROM search system and the other was to provide insights 
into potential improvements in the interface design and user training programs. 

Pat Ensor 8 conducted a survey at Terre Haute Libraries of Indiana State University 
while the ABI/INFORM Ondisc CD-ROM database was available in a two-month free trial. 
Respondents consisted of users in business schools and non-business schools. User 
satisfaction with search results made ISU decide to subscribe to the product. 

The study undertaken during the second half of February 1988 by Peter M. Lepoer 
and Carol A. Mularski 9 investigated the reactions of both new and experienced users to the 
Compact Cambridge MEDLINE on CD-ROM available in the Ohio State University Health 
Sciences Library. Findings revealed that most patrons seemed to be satisfied with the 
availability of the system even though very few users were able to formulate totally efficient 
search statements due to lack of training or experience. 

Some of the literature compared the use of a printed index versus a CD-ROM index. 



The research done in Oakland University's Kresge Library by Kim Schultz and Kristine 
Salomon 10 asked students to compare CD-ROM and printed versions of ERIC, PsycLit, and 
the Business Periodicals Index. The participants were twenty-five business, nine psychology, 
and eight education students. Their results showed that CD-ROM versions of those indexes 
were faster and easier to use; besides, the overwhelming majority of the students surveyed 
would use CD-ROM indexes instead of printed indexes next time when they had to do their 
searches. 

Prosperous as they are, CD-ROM bibliographic indexes are not always the winner. 
An experiment performed by Carol Reese 11 proved this fact. Seventeen students from 
Brookdale Community College, Lincroft, New Jersey were divided into two groups. They 
were asked to research the same questions either on InfoTrac II CD-ROM database or in 
the manual index, Readers' Guide to Periodical Literature, to see which index provided 
them with better access to relevant information. The statistic data indicated "Readers' 
Guide" to be more effective. 

Articles comparing different products also exist. Gillian Allen's 12 "Patron Response 
to Bibliographic Databases on CD-ROM" is an example. This study examined patron 
response to and success in using six Wilsondisc bibliographic databases and the Info Trac 
General Periodicals Index in the Undergraduate Library at the University of Illinois at 
Urbana-Champaign. The methodology incorporated observation of patrons' searches and 
questionnaires completed by the participants. Regarding patron satisfaction with search 
results, the median responses were positive. In terms of the need for CD-ROM instruction, 
"agree strong" that training was not required was the median response selected. It was also 



observed that InfoTrac searchers employed more search statements and had more failures 
compared to Wilsondisc users. 

CD-ROM REFERENCE SERVICES AT SUNY/ALBANY 
Background 

The University at Albany, State University of New York, is one of four university 
centers in New York State's public university system with over 16,800 students, of which 
approximately 5,000 are graduate students. The full-time faculty consists of 620 members, 
and they teach both graduates and undergraduates. The Main Library serves a diverse 
student population plus an array of faculty, staff, and community members. 

In February 1987, the Library obtained its first CD-ROM product, ERIC, on trial. 
Due to a shortage of funds, when the trial period expired, the CD-ROM was returned to the 
vendor. In March 1988, hardware was installed to provide end-users with access to ERIC 
and PsycLit on CD-ROM. The CD-ROM facilities were housed in the basement of the 
Library in the same area as the online search service. By Fall 1989, three more databases 
were on subscription and the number of terminals was increased to four. In addition, the 
CD-ROM area was moved upstairs to the location where it is now. In the summer of 1990, 
the number of workstations were expanded to eight and they were networked to meet the 
large demand in the use of CD-ROMs by end-users. In January 1991, the Library had 
eleven workstations in total, of which three were stand-alone. At the time of this survey, 
databases available on the network were ABI/INFORM, Academic Index, PAIS, Newspaper 
Abstracts Ondisc, ERIC, CaDiLaC, GPO Monthly Catalog, MLA Bibliography, PsycLit, 



Social Science Index, and Sociofile. Fifty-four databases could be accessed at stand-alone 
workstations. Since the new technology is strongly favored by the users, the Library has 
committed heavily to the rse of CD-ROM products. The total uses of CD-ROMs during 
the period of July 1991 to October 1992 was ninety-five thousand times, and more than ten 
thousand times in November 1992. 
CD-ROM Service And User Assistance 

The CD-ROM area is located right behind the Reference Desk. The workstations 
have been set up in the form of a horseshoe, the three stand-alone terminals are on the 
right, and the help desk is in the middle. The service is available to all library users any 
time the Library is open. Throughout the academic year, products may be used between 
8:00 a.m. and 11:00 p.m., Monday through Thursday, 8:00 a.m. to 10:00 p.m. on Friday, 8:00 
a.m. to 8:00 p.m. on Saturday and from noon to 11:00 p.m. on Sunday. User assistance is 
usually available from 9:00 a.m. to 10:00 p.m. Monday through Thursday, and 9:00 a.m. to 
6:00 p.m. on Friday and on a limited basis on weekends. The CD-ROM reference librarian, 
who is a librarian or a graduate assistant from the masters program of the School of 
Information Science and Policy at SUN Y/ Albany, provides one-on-one, immediate 
assistance to anyone who needs help while conducting a search on CD-ROM. Assistance 
varies from putting more paper in the printer to giving brief instruction in how to conduct 
a search, depending on the kind of help the user needs and how much time the librarian has 
to devote to one user. When the CD-ROM reference librarian is not on duty, librarians 
working at the Reference Desk provide assistance to the user. 
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Reservations 

Because of the heavy use of the workstations, the user may reserve time to search 
CD-ROM databases. Reservations are strongly recommended during the busy periods of 
the semester. Stations that are not reserved are available on a first-come first-serve basis. 
Reservations may be made in person or by telephoning the Library Reference Desk. 
Reservations shculd be made in the appropriate notebook in the CD-ROM area. One 
notebook serves workstations on the network and one serves stand-alone workstations. 
When signing up a stand-alone workstation, a specific database should be reserved as well. 
Reservations are limited to one hour per day for up to two hours per week and may be 
made up to two weeks in advance. A user arriving for his/her appointment is entitled to 
ask anyone using the terminal to vacate. 
Documentation 

Thesauri and user manuals specific to the databases offered are kept on the CD- 
ROM help desk. Printed database search guides are displayed on a rack to the right of the 
entrance of CD-ROM area. These search guides are created by SUNYA librarians. The 
content of the guide includes a description of the database, time period covered, 
corresponding printed and online sources, thesaurus information and an explanation of the 
function keys and fields. The simple search procedures, logic operators, truncation symbol, 
method for displaying records, information on printing and downloading, how to obtain on- 
line heip and a sample record are also introduced. Users, especially first-time users, are 
encouraged to consult them. 



Workshops (Training Sessions) 

The Library has offered CD-ROM training sessions for about three years. During 
the Fall semester 1992, forty-seven workshops were held at the Library. They began with 
the first session on September 9 and ran through December 3. Instruction was offered for 
the following databases: ABI/Inform, Business Periodicals Index, ERIC, Medline, MLA 
Bibliography, PAIS, PsycLit, Science Citation Index, and Sociofile. In addition, a General 
Introduction workshop was provided. The aims of the workshop are to increase user 
effectiveness and reduce the workload of novice user instruction on reference staff. These 
workshops teach end-users to search a database efficiently in order to develop a customized 
bibliography. End-users can also learn how to print retrieved citations or download them 
to disks. The instruction program is offered free of charge and is taught by SUNYA 
librarians. Each session lasts approximately one hour. Workshops on ERIC, PAIS, PsycLit, 
Sociofile are held at the CD-ROM workstations; while the General Introduction workshop 
and workshops on the other databases are held in a classroom in the Library. Class size is 
limited to provide end-users with individual attention. First-time users and users who need 
extensive help are encouraged to enroll in a workshop. 
Course-Integrated Bibliographic Instruction 

In addition to the CD-ROM reference librarian, database search guides, workshops, 
the CD-ROM course-integrated bibliographic instruction is available, too. This type of 
bibliographic instruction program is offered at the request of faculty, and it provides group 
instruction to students in accordance with specific subject classes. The content of each 
instruction varies depending on the requirements set up by the course instructor rmd the 
subject being taught. The degree to which CD-ROM searching techniques are taught is 
based on its applicability to each individual subject. 
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THE SURVEY 
Purpose of the Survey 

The purpose of the present study was to 

1. Investigate the problems the CD-ROM users encountered 
while they conducted their searches. 

2. Determine users' needs with regard to the improvement in 
the CD-ROM databases they used. 

3. Find out the preferred instruction programs they wanted. 

The results will enable us to find the ways to enhance the services, as well as the systems, 
in order to maximize successful end-user searching on CD-ROM. 
The Survey Instrument 

The survey consisted of twelve questions. This two-page instrument solicited 
information on institutional affiliation, previous searching experience, purpose(s) for doing 
the search, CD-ROM database(s) used, how to learn to use the system(s), search strategies, 
problem(s) encountered, evaluation of the system(s) used, search results, priorities for 
improving the system(s), and instructional methods recommended. An open-ended question 
was included to ask for users' comments and suggestions. The questionnaire was designed 
to be anonymous and was kept brief and simple in order to encourage people to participate. 
Conducting the Carvey 

The survey was conducted from November 4 to November 10, 1992. This time frame 
fell in the middle of the semester and contained no holiday breaks. The researcher spent 
approximately thirty-four hours, in blocks of two to f rx i! r J^rs, at various times of the day 
including evenings and weekends, administering the survey. Like many other surveys that 



attempted to evaluate user response to CD-ROM products, the researcher was not able to 
gather a truly random sample of the target population. Therefore, the data was collected 
from the first 200 non-repeat SUNY/Albany affiliated users. During the designated survey 
blocks, green signs which stated "CD-ROM Survey in Progress" were put on the help desk 
and at each workstation. As a user completed his/her search(es) and was ready to leave, 
the researcher approached the user, explained the nature of the survey and asked him/her 
to voluntarily fill out the questionnaire. Library staff and non-SUNY/ Albany affiliates were 
eliminated. 242 users were approached, of which 18 were non-SUNYA, and 24 were not 
willing to participate because of time constraints. Since the users surveyed did not see the 
questionnaire until they finished their searches, the responses given were absolutely their 
own perception about the service. 



SURVEY RESULTS 

A profile of the 200 respondents shows that r majority were either graduate students 
(52%) or undergraduates (45.5%). The remainder was comprised of SUNYA faculty (2%) 
and staff (0.5%). The total and the percentage of each category under the user status are 
tabulated in Table 1. 



Table 1 
CD-ROM Users by Status 





Faculty 


Graduate 


Undergrad 


Staff 




Total 


4 


104 


91 


1 


200 


% 


2 


52 


45.5 


0.5 


100 
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In the area of previous search experience, 25 respondents(12.5%) were first-time 
users. Of the 175 people who had prior experience, 67 (33.5%) have used CD-ROM 
databases 1-5 times and 40 (20%) of them were undergraduates: 35 people(17.5%) indicated 
that they have searched CD-ROM databases 6-10 times; 17 persons (8.5%) have used the 
products 11-15 times, and 56 people (28%) claimed that they have experience over 15 times, 
of which 37 (18.5%) were graduate students. Overall, every faculty member and staff person 
who filled out the survey had CD-ROM search experience more than 15 times, 47% of the 
graduate students had prior experience more than 10 times, while the majority of the 
undergraduates (64%) had CD-ROM experience fewer than 6 times. The level of 
experience by user status revealed that the faculty and staff were the most experienced, 
followed by graduates, with undergraduate students far less experienced. Table 2 shows the 
relationship between the users and their prior CD-ROM experiences. 



Table 2 

CD-ROM Users and Their Prior Experiences 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 


0 




7 


18 




25 


12.5 


1-5 




27 


40 




67 


33.5 


6-10 




21 


14 




35 


17.5 


11-15 




12 


5 




17 


8.5 


over 15 


4 


37 


14 


1 


56 


28.0 


Total 


4 


104 


91 


1 


200 


100 



With regard to the purpose of doing CD-ROM searches, participants could have 
selected multiple responses. The largest percentage of users (78%) were using the CD- 
ROMs for a "research paper/project," 47% checked "class assignment," 13% worked on 
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"thesis/dissertation," 10.5% looked for "professional information," and 5.5% specified 
"other" including personal interest, assistantship, learning how it works, interview, 
publication, etc. Although the responses indicate a varied use of the workstations, finding 
information for research paper/project was the primary purpose of all users no matter what 
status they were; while "class assignment" was the secondary reason of both graduates and 
undergraduates. Table 3 summarizes the findings. 



Tabic 3 

Purpose(s) of searching CD-ROM by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 


Class 

assignment 




50 


44 




94 


47 


Research 
paper/project 


4 


80 


71 


1 


156 


78 


Professional 
information 


1 


16 


3 


1 


21 


10.5 


Thesis/Dissert 
ation 


1 


21 


4 




26 


13 


Other 
purpose (s) 




6 


5 




11 


5.5 



Note: Sometimes the user did more than one search for different purposes, so the total searches is 
over 200, and the sum of the percentages is over 100. 



During the duration of the survey, a total of 23 CD-ROM products were involved in 
the study. Of the 200 users, 52 (31 graduates, 19 undergraduates and 2 faculty) searched 
more than one database to retrieve information. The most commonly used products were 
Academic Index (50 users or 18.4%), ERIC (43 users or 15.8%), PsycLit (38 users or 
14.0%) ABI/INFORM (28 users or 10.3%) and Newspaper Abstracts Ondisc (21 users or 
7.7%). The least used were Library Literature, Population Statistics, County Business 
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Patterns and Congressional Masterfile; each of these files were accessed by one user only. 
In general, databases on stand-alone workstations were used far less than those networked. 
When broken down by user status, graduate students used ERIC the most, while 
undergraduates used Academic Index most frequently. Table 4 on the next page indicates 
the status of users for each use of the individual CD-ROM products. 

When asked to check all the methods by which they had learned to use the system, 
52.5% of the users said that they learned by "trial and error," 45.5% iearned from librarians, 
26.5% figured out how to work on the product by means of reading the "help screen," 
21.5% were taught by friends/classmates, 17.5% learned by attending workshops and 17% 
consulted handouts(database search guides). Upon close examination, it can be seen that 
the percentages for individual patron categories do not follow closely with those of the 
entire group. In both the graduate and undergraduate categories, over 50% of each user 
population tended to try by themselves, and over 40% relied on the services of the librarian, 
while 100% of the faculty and staff received assistance from d librarian and only 25% of the 
faculty learned by "trial and error." The percentage of graduate students who used the help 
screen, handouts and attended workshops is almost twice or more than twice as those of 
undergraduates for each of these means of learning. "Friends and classmates" played a role 
in assisting the undergraduate population. Table 5 summarizes ihe instructional aids 
employed by users by their status. 
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Table 4 

Use of Individual CD-ROM Database by User Status 
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Disclosure 
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3 
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1.8 
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1 


2 
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1.1 


NTDB 




2 
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0.7 


SCI 




2 


1 
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1.1 


*Pop. Statistics 




1 






1 


0.4 


*Co. Bus. Pat. 




1 






1 


0.4 


*Cong. Master. 




1 






1 


0.4 



Note: *Acad. Index = Academic Index 

*Bio. & Agri. = Biological & Agricultural Index 

*Disser. Ab. = Dissertation Abstracts Ondisc 

*Pop. Statistics = Census Data on Population and Housing 

*Co. Bus. Pat. = County Business Pattern 

*Cong. Master. = Congressional Masterfile 



14 



Table 5 

Means Employed to Learn CD-ROM Database(s) by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total # 
of use 


** % 




Total 


*% 


Total 


* % 


Tot?i 


* % 


i oiai % 






Help Screen 


0 


39 


38 


14 


15 




53 


26.5 


Handouts 


1 


25 


22 


21 


11 


10 




34 


17 


Librarian 


4 


100 


44 


42 


42 


46 


1 100 


91 


45.5 


Workshop 


0 


26 


25 


9 


10 




35 


17.5 


Friends 


0 


20 


19 


27 


30 




47 


23.5 


Trial & error 


1 


25 


57 


55 


47 


52 




105 


52.5 



Note: * the percentage of individual group 
** the percentage of the total users 



For people who had no prior CD-ROM experience, librarians were the most 
commonly used instruction source. The more experienced the users were, the more they 
used the help screen, took advantage of the workshop, asked friends or classmates for help 
and learned to figure out how the system works by "trial and error. 1 ' The percentage of 
people who have searched CD-ROMs over 15 times and consulted the handouts (database 
search guides) was more than twice as tho$e of other less experienced groups. It surprised 
the researcher that evea though they are the most experienced, over 45% of them still 
relied on the instruction provided by the librarians, instead of attending the workshops. 
Table 6 on the following page reveals the relationship between the avenues of learning the 
systems and the user's prior experience. 
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Table 6 

Means of Learning CD-ROM Database by User Prior Experience 





0 


1-5 


6-10 


11-15 


15+ 


Total 
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HelD Screen 
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6 


24 


19 


28 


7 


20 


5 


29 


16 


29 


53 


Handouts 


3 


12 


9 


13 


3 


9 


2 


12 


17 


30 


34 


Librarians 


14 


56 


31 


46 


15 


43 


5 


29 


26 


46 


91 


Workshop 


3 


12 


12 


18 


7 


20 


6 


35 


7 


13 


35 


Friend 


3 


12 


18 


27 


11 


31 


5 


29 


10 


18 


47 


Trial & error 


10 


40 


32 


48 


18 


51 


11 


65 


34 


61 


105 



The next question that the researcher asked was "Did you spend time developing your 
search strategy?" The purpose of this question was to find out whether people thought 
about a search strategy before sitting down at the computer. If they answered "yes," then 
they were asked how they formulated their searches. Some people did more than one 
search, so they gave multiple responses. Of the 200 participants, 26 (13%) did not know 
what search terms they should use. They were 9% of the graduate students and 19% of the 
undergraduates. The overwhelming majority (70%) thought about the key words to be 
searched in their head. Only 12.5% thought about all the possible search terms and put 
them down on paper. Fifteen percent looked up the thesaurus on the CD-ROM database. 
Faculty and graduate students used the thesaurus more than undergraduates and staff did. 
Very few people (7%) said they used Boolean logic operators to formulate their search 
statements on paper. They consisted of 9% of the graduates and 5% of the undergraduates. 
Compared with other user groups, the most experienced had more control over the search 
terms they should use. In addition, 25% of them put all the possible key words on paper 
before doing their searches and 14% used Boolean logic operators to formulate their search 
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statements. The groups that had searched CD-ROMs more than 10 times made more use 
of the on-line thesaurus than those who were less experienced. Table 7 presents the 
findings by user status, and Table 8 shows the search strategy employed by user groups with 
different levels of experience. 



Table 7 

Search Strategy Use by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


* % 




Total 


% 


Total 


% 


Total 


% 


Total % 






No strategy 
used 




9 


9 


17 


19 




26 


13 


Key words in 
head 


3 


75 


71 


68 


65 


71 


1 100 


140 


70 


Possible terms 
on paper 


1 


25 


14 


13 


10 


11 




25 


12.5 


Online 
Thesaurus 


1 


25 


23 


22 


6 


7 




30 


15 


Boolean logic 
operators 




9 


9 


5 


5 




14 


7 



Note: * the percentage of total respondents 

Since some users did more than one search, the total searches is over 200, ana the sum of 
the percentages is more than 100. 



Table 8 

Search Strategy Used by User Prior Experience 





0 


1-5 


G-10 


11- 


15 


15+ 


Total 




Total 


% 


Total 


% 


Total 


% 


Total 


% 


Total 


% 




No strategy 
used 


5 


20 


14 


21 


3 


9 


4 


24 




26 


Key words in 
head 


17 


68 


45 


67 


29 


83 


11 


65 


38 


68 


140 


Possible terms 
on paper 


1 


4 


6 


9 


2 


6 


2 


12 


14 


25 


25 


Online 
Thesaurus 


3 


12 


9 


13 


2 


6 


4 


24 


12 


21 


30 


Boolean logic 
operators 


2 


8 


1 


1 


3 


9 




8 


14 


14 
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The rest of the questions on the survey is the core of the investigation. By asking 
"What factors influenced your search?", the researcher intended to find out the problems or 
difficulties the patrons had while they conducted their searches. Forty-nine patrons (24.5%) 
said they had to wait to use the terminals. The same portion of patrons claimed they had 
to make an appointment beforehand. These were common problems in all categories of 
users. Thirty-nine patrons (19.5%) revealed that they did not know what databases they 
should choose until they asked the librarian. This was reported by 15 (14%) of the 
graduate students and 24 (26%) of the undergraduates. Forty-one or 20.5% of the total 
patrons did not know how to retrieve relevant information more effectively by using Boolean 
logic, thesaurus, etc. Twenty-five percent of each category of the faculty and 
undergraduates had this difficulty, while only 16% of the graduate students had this 
problem. Forty-four (22%) of the surveyed users indicated that they were not sure whether 
the citation retrieved was relevant to their search topics or not. Faculty and staff did not 
have any uncertainty about the information they retrieved. Twenty-one or 20% of the 
graduates and 23 or 25% of the undergraduates had trouble in judging the relevance of the 
retrieved citations. This phenomenon is not unusual since the patrons had not read the 
articles or books by the time they filled out the survey, it was not easy to see the relevance 
by just looking at the citations. It was a good thing that the computers never went down 
during the period of the survey, but 6 (3%) of the patrons pointed out that the screen froze 
while they were doing their searches. The detailed statistics of the problems encountered 
by user status is provided in Table 9. 



ERLC 



18 



9^> 



Table 9 

Problems Encountered by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 




Total % 


Total 


% 


Total 


% 


Total % 






Have to wait to 
use 


2 50 


20 


19 


26 


29 


1 100 


49 


24.5 


Have to make 
appointment 


1 25 


26 


25 


22 


24 




49 


24.5 


Unfamiliar with 
keyboard 




1 


1 


7 


8 




8 


4 


The screen 
froze 




4 


4 


2 


2 




6 


3 


Printer 
problem 




10 


10 


13 


14 


1 100 


24 


12 


Computer 
went down 














Help key not 
useful 




6 


6 


5 


5 




11 


5.5 


No immediate 
assistance 


1 25 


4 


4 


5 


5 




10 


5 


Database to 

cnoose 




15 


14 


24 


26 




39 


19.5 


Not enough 
search time 




18 


17 


11 


12 




29 


14.5 


Using Boolean 
logic 


1 25 


17 


16 


23 


25 




41 


20.5 


Citation 
uncertainty 




21 


20 


23 


25 




44 


22 



It was also observed that the problems of waiting to use terminals, making 



reservations, and being uncertain about the usefulness of the information found were 
universal no matter how much CD-ROM experience the users had before. Nevertheless, 
only users with prior experience less than 11 times had the keyboard problem. In addition, 
users with less experience had more problems of not being able to get immediate assistance 
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from the librarian, had more difficulties in selecting the appropriate databases they needed, 
and had more trouble in using Boolean logic or the online thesaurus. Table 10 tabulates 
the relationship between the problems confronted and user status under the categories of 
prior experience. 



Table <0 

Problems Encountered by User Prior Experience 





0 


1-5 


6-10 


11- 


15 


15+ 


Total 




Total 


% 


Total 


% 


Total 


% 


Total 


% 


Total % 




Have to wait to 
use 


5 


20 


16 


24 


10 


29 


5 


29 


13 23 


49 


Have to make 
appointment 


5 


20 


18 


27 


9 


26 


2 


12 


15 27 


49 


Unfamiliar with 
keyboard 


2 


8 


6 


9 








8 


The screen 
froze 


2 


8 


2 


3 


2 


6 






6 


Printer 
problem 


2 


8 


6 


9 


6 


17 


6 


35 


4 7 


24 


Computer 
went down 














Help key not 
useful 




3 


4 


2 


6 


2 


12 


4 7 


11 


No immediate 
assistance 




7 


10 


2 


6 




1 2 


10 


Database to 
choose 


7 


28 


25 


37 


4 


11 




3 5 


39 


Not enough 
search time 


2 


8 


14 


21 


6 


17 


1 


6 


6 11 


29 


Using Boolean 
logic 


6 


24 


18 


27 


10 


29 


2 


10 


5 9 


41 


Citation 
Uncertainty 


6 


24 


15 


22 


7 


20 


5 


29 


11 20 


44 
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In order to explore the major drawbacks of the systems available, the researcher 
&st*d the patrons to make evaluation of the products based on the first database they used. 
Just over twenty-one percent (21.5%) complained about the lack of flexibility in combing 
search statements; 15% thought the systems were slow, and 12.5% felt that the information 
on the help screen was not specific enough. In regard to the content-related problems, 20% 
considered the years covered by the systems not long enough. Another complaint was the 
lack of abstracts. Table 11 displays the findings by user status. The breakdown of problems 
related to individual database is given in Table 12. 



Table 11 

System Problems Found by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 


Slow Response 
time 


2 


17 


10 


1 


30 


15 


Lack of flexibility 




24 


19 




43 


21.5 


Inadequate menu 
description 




7 


5 




12 


6 


Need to switch 
disks 




3 


2 




5 


2.5 


| Date range not 
8 long enough 




17 


23 




40 


20 


| Does not have 
fl abstracts 




10 


8 




18 


9 


| No listing of titles 




7 


6 




13 


6.5 


Marking problem 




9 


7 




16 


8 


Error message 
not informative 




3 


3 




6 




Help screen not 
specific 




12 


13 




25 


12.5 
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Table 12 

System Problems by Individual Database 





A 


B 


C 


D 


E 


F 


G 


H 


I 


J 


PsycLit 


3 


6 


1 


2 


5 


2 


5 


3 






5 


Medline 


1 


1 




3 














Acad. Index 


1 


7 


1 




7 


2 


1 








2 


Bio. & Agri. 






1 














2 


ABI/INFORM 


8 


4 


1 




5 




1 


2 




3 


MLA 


1 


2 






1 


2 


1 


1 






ERIC 


7 


10 


3 




5 


3 


2 


2 


2 


8 


SSI 






2 




1 


1 










Sociofiie 


1 


4 






1 


3 


2 


1 


1 




BPI 




1 






1 


2 


1 


2 






Art Index 










1 










1 


Disser. Ab. 






1 




2 


1 




1 




1 


News. Ab. 


6 


4 


2 




3 


1 




3 


1 


2 


Lib. Lit. 










1 


1 




1 






PAIS 










2 












CaDiLaC 




2 






2 












Disclosure 


1 








1 








1 


1 












1 













NTDB 




1 


















SCI 






















Pop. Statistics 










1 












Co. Bus. pat. 


1 


1 


















Cong. Master. 























Note: Problem A: Slow response time Problem F: Does not have abstracts 

Problem B: Lack of flexibility Problem G: No listing of titles 

Problem C: Inadequate menu description Problem H: Marking problem 

Problem D: Need to switch disks Problem I: Error message not informative 

Problem E: Date range not long enough Problem J: Help screen not specific 
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The researcher was also interested in the problems of specific product lines, so the 
database used was classified into four groups: SilverPlatter, Wilsondisc, UMI and "other." 
The results showed that UMI products had the most serious problem of slow response time. 
The most inadequate description of menu items was produced by Wilsondisc. Additionally, 
this product line had the greatest problem of not providing abstracts, and not allowing the 
searchers to mark the items they were interested in and then print them out at one time. 
Another annoying drawback of Wilsondisc products was that the systems did not allow the 
users to view a listing of titles of the citations retrieved on one screen. While problems like 
lack of flexibility, short of date range, non-specific help screen were commonly shared by 
all product lines. Table 13 shows the users' reactions to the systems by product lines. 



Table 13 

System Problems by Product Lines 





SilverPlatter 
(90 users) 


Wilsondisc 
(20 users) 


UMI 
(45 users) 


Other 
(45 users) 




Total 


% 


Total 


% 


Total 


% 


Total 


% 


Slow response 
time 


12 


13 


1 


5 


14 


31 


3 


7 


Lack of flexibility 


21 


23 


3 


15 


8 


18 


11 


24 


Inadequate menu 
description 


4 


4 


3 


15 


4 


9 


1 


2 


Need to switch 
disks 


5 


6 








Date range not 
long enough 


14 


16 


5 


25 


10 


22 


11 


24 


| Does not have 
1 abstracts 


8 


9 


6 


30 


2 


4 


2 


4 


I No listing of titles 


9 


10 


2 


10 


1 


2 


1 


2 


a Marking problem 


6 


7 


4 


20 


6 


13 




| Error message 
I not informative 


4 


4 




1 


2 


1 


2 


I Help screen not 
I specific 


13 


14 


3 


15 


6 


13 


3 


7 
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The next thing the researcher questioned the users about was their search results. 
A little more than thirty-three percent (33.5%) of the respondents said they found more 
citations than they expected; 32% indicated that they retrieved about the number of citations 
they wanted, and 31.5% got fewer citations than they needed. Six or 6% of the graduate 
students and 6 (7%) of the undergraduates did not find any information on their topics. 
Evaluation of the data leaves the impression that the more experienced did not necessarily 
get better search results than the less experienced. In addition, there was no guarantee that 
users who developed their search strategies with Boolean logic or thesaurus yielded better 
search results. This was because users' needs varied. Some might need thorough searches 
that did not miss anything, and others might just want a few precise citations. The 
information provided by the databases, such as the time period covered, the journals/books 
included, also had an influence on users' search results. Tables 14, 15, and 16 present the 
relationship between users and their search results by user status, user prior experience, and 
search strategies applied respectively. 



Table 14 
Search Results by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 




Total % 


Total 


% 


Total 


% 


Total % 






More than 
expected 


1 25 


27 


26 


39 


43 




67 


33.5 


| About the 
j number 




38 


37 


25 


27 


1 100 


64 


32 


| Fewer than 
expected 


3 75 


34 


33 


26 


29 




63 


31.5 


No citation 
on topic 




6 


6 


6 


7 




12 


6 
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Table 15 

Search Results by User Prior Experience 





0 


1-5 


6-10 


11-15 


15+ 


Total 




Total 


% 


Total % 


Total % 


Total % 


Total % 




More than 
expected 


9 


36 


24 36 


10 29 


8 47 


16 29 


67 


About the number 
expected 


9 


36 


21 31 


14 40 


5 30 


15 27 


64 


Fewer than 
expected 


5 


20 


17 25 


13 37 


2 12 


26 46 


63 


No citation on 
I topic 


2 


8 


6 9 






4 7 


12 



Table 16 

Search Result by Search Strategy Employed 





NO 


S1 


S2 


S3 


S4 


Total 




Total 


% 


Total 


% 


Total 


% 


Total 


% 


Total 


% 




More than 
expected 


8 


31 


48 


34 


9 


36 


11 


37 


6 


43 


83 


About the number 
expected 


10 


38 


44 


31 


8 


32 


11 


37 


4 


29 


76 


I Fewer than 
| expected 


6 


23 


45 


32 


7 


28 


6 


20 


5 


36 


69 


I No citation on 
|| topic 


2 


8 


5 


4 


1 


4 


5 


17 


1 


7 





Note: No : No strategy employed 

$1 : Think about key words in head 

52 : Put all the possible search terms on paper 

53 : Look up Thesaurus on CD-ROM 

54 : Use Boolean logic to formulate search statements 



25 



2d 



Question 10 correlated with question 8. It offered users an opportunity to rank the 
three most important ways they would improve the database they were using. A list of 
possible methods and an "Other" option in which to specify one of their choices was given. 
The results of that tally are arranged by user status, product line and individual database 
in Tables 17, 18, and 19. Generally speaking, making the system accessible from offices and 
homes was the first choice of 2 of the faculty (50%), 33 graduate students (32%), and 50 
(25%) of the total population surveyed; while including more journals/books was the first 
concern of 22 (24%) undergraduates and the second priority of faculty members and 
graduate students. When adding up the numbers of priorities 1, 2 and 3 together for each 
option, we could see the rank of product improvement priority is: 

1. "Include more journals/books" 53% 

2. "Make it accessible from offices and homes" 52.5% 

3. "Expand the range of years covered" 43% 

4. "Make the Help Screen be more specific" 29% 

5. "Make the information covered more current" 28.5% 

6. "Provide abstracts" 21% 

7. "Change the search procedures" and "Other" 17% 

8. "Include fewer journals/books" 3% 

Table 17 shows the priority order concerning the improvements to be made by user status. 
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Table 17 

System Improvements by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 




Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


*Total 


Make info, 
more current 




11 6 8 


11 12 8 


1 


22 19 16 


57 


Expand date 
range 


1 1 


17 16 11 


11 16 12 




29 33 23 


85 


More 

journals/books 


1 


14 25 18 


22 18 8 




36 44 26 


106 


| Fewer 

I journals/books 




1 


4 1 




4 2 


6 


| Provide 
| abstracts 




7 8 6 


6 7 8 




13 15 14 


42 


| Change search 
| procedures 




7 5 3 


3 3 13 




10 8 16 


34 


| Specific help 
I screen 




8 10 6 


13 12 8 


1 


21 22 15 


58 


| Accessible 
| from home 


2 1 


33 13 15 


15 9 17 




50 22 33 


105 


I Other 


1 1 


6 6 4 


10 4 1 


1 


18 10 6 


34 



Note: * The number of times the improvement was selected by all the respondents. 



According to the responses of question 8, we know each product line has its own 
problems. The researcher again divided the databases used into four groups based on the 
producers. The improvements the users would make for each product line is tabulated in 
Table 18. The three priorities selected by SilverPlatter products users were the same as 
those chosen by the total respondents. Wilsondisc products users ranked "include more 
journals/books," "provide abstracts," and "make it accessible from offices and home" as the 
three improvements they felt were most important. As for UMI products users, "include 
more journals/books" was the most important, "expand the range of years covered" was the 
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secondary concern, and the third thing to do to improve the index was "make it accessible 
from offices and homes." Users of other product lines checked the same items as UMI users 
did but ranked them in a different way. Table 19 demonstrates the priority of the 
improvements by individual database. 



Table 18 

System Improvements by Product Lines 





SilverPlatter 
(90 users) 


Wilsondisc 
(20 users) 


UMI 
(45 users) 


Other 
(45 users) 


Priority order 


1 




O 
O 


1 


2 


3 


1 


2 


3 


1 


2 


3 


Make info, more 
current 


1 4 


a 
o 


ft 




1 


3 


4 


5 


3 


7 


5 


4 


Expand date 
range 


13 


13 


9 


3 


2 


1 


4 


12 


8 


9 


6 


5 


More 

journals/books 


11 


18 


11 


4 


9 


2 


14 


8 


5 


7 


9 


8 


Fewer 

journals/books 




2 


1 




1 


2 


Provide 
abstracts 


5 


3 


6 


6 


2 


3 


1 


5 


2 


1 


5 


3 


I Change search 
| procedures 


5 


3 


7 


1 


5 


3 


4 




2 


4 


| Specific help 
B screen 


9 


12 


7 


4 


1 


2 


6 


4 


2 


2 


5 


4 


| Accessible from 
I home 


28 


11 


12 


2 


3 


5 


8 


3 


11 


13 


4 


5 


[ Other 


9 


7 


3 


1 


1 


1 


2 


2 


2 


6 



28 



Pi 



Table 19 

System Improvements by Individual Database 





PsycLit 


Medline 


Acad. Index 


Bio. & Agri. 


ABI/INFORM 


Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


| Make info, 
more current 


3 4 2 


1 


4 2 3 




1 4 3 


Expand date 
range 


3 4 7 


1 


7 5 3 




1 7 2 


More 

journals/books 


4 5 6 


1 2 


5 7 4 




9 2 3 


Fewer 

journals/books 


2 




2 






Provide 
abstracts 


1 1 1 




1 5 3 




1 3 1 


Change search 
procedures 


3 12 




2 4 


1 


3 12 


Specific Help 
screen 


2 5 3 




2 4 3 


1 1 


1 1 2 


Accessible from 
home 


13 2 4 


1 


10 1 5 




6 2 5 


Other 


3 2 


2 


4 


1 1 


1 1 1 
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Table 19 continued 





MLA 


ERIC 


SSI 


Sociofile 


BPI 


Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


Make info, 
more current 


1 1 


5 3 1 




2 1 1 


2 


Expand date 
range 


1 


7 S 1 


1 


1 2 


1 


More 

journals/books 


1 4 


4 5 3 


1 1 


1 4 1 


2 3 


Few 

journals/books 




1 








Provide 
abstracts 


3 1 2 


2 1 1 


1 


1 1 2 


2 1 


Change search 
procedures 




2 2 4 




1 




Specific Help 
screen 


2 


4 7 4 


1 


1 




| Accessible from 
| home 


1 4 


13 6 4 




1 1 3 


2 2 1 


| Other 




2 5 3 




2 


1 j 
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Table 19 continued 





Art Index 


Disser. Ab. 


News. Ab. 


Lib. Lit. 


PAIS 


Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


Make info more 
current 






3 1 




1 


Expand date 
range 


1 1 


1 4 


3 4 2 


1 


1 2 


■ ■ ■ _ 

| More 

| journals/books 


2 


2 1 


3 5 2 


1 


111 


| Fewer 

| journals/books 






1 






Provide 
abstracts 




1 


1 1 


1 


2 


Change search 
procedures 




1 1 


112 






Specific Help 
screen 


1 1 


1 


4 3 




1 


Accessible form 
homes 




1 


1 1 6 




1 1 


Other 




1 


1 1 
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Table 19 continued 





CaDiLaC 


Disclosure 


GPO 


NTDB 


SCI 


Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


Make info, 
more current 


12 1 


2 


1 






Expand date 
range 


1 2 




1 




1 


More 

journals/books 


1 1 


1 1 


1 


1 


1 


Fewer 

journals/books 












Provide 
| abstracts 






1 






I Change search 
| procedures 












I Specific Help 
| screen 




1 


1 






I Accessible from 
I home 


1 1 


1 


1 




1 


| Other 


1 


1 
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Table 19 continued 





Population 
Statistics 


County Bus. 
Pattern 


Congression 
Masterfile 


**Total 


♦Total J 


Priority order 


1 2 3 


1 2 3 


1 2 3 


1 2 3 




Make info, 
more current 


1 






22 19 16 


57 I 


Expand date 
range 


1 






29 33 23 


85 


More 
| journals/books 


1 


1 




36 44 26 


106 


Fewer 

journals/books 








4 2 


6 


Provide 
abstracts 








13 15 14 


42 


Change search 
procedures 








10 8 16 


3< 


I Specific Help 
I screen 






1 


21 22 15 


58 


I Accessible form 
0 homes 




1 


1 


50 22 33 


105 


I Other 








18 10 6 


34 



Note: ** The total number of times the improvement was selected for all databases for each priority 
1,2, 3. 

* The number of times the improvement was selected by all respondents. 



A variety of means regarding products improvements were specified in the "other" 
option. Among those the most commonly mentioned were: having more terminals 
available, indicating whether or not the Library has the journal/book indexed in the 
database, loading call numbers into the software, providing full text, and allowing more 
search time. The rest of the opinions are listed in Appendix A. 

The last question on the survey was pertinent to CD-ROM instructional programs. 
The purpose of this question was to find out what the CD-ROM user want, thus vhe 
librarians would be able to try to offer the services that tailor to their needs. In general, 
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the most commonly recommended models of instruction were: "Computer-Assisted 
Instruction," (CAI) 89 users or 44.5%; "individual instruction on demand," 80 users or 40%; 
"written guides with sample search questions," 77 users or 38.5%, and "workshops to learn 
how to choose the right database, select the search terms, use Boolean logic, formulate 
search strategies, etc.," 72 users or 36%. Only 13 users (6.5%) thought "no formal 
instruction is needed." If we break down the data by user status, we can see that "CAI" was 
the most strongly desired program by both graduate students and undergraduates. 
Individual instruction on demand was preferred by faculty and undergraduates vhile 
workshops concentrating on searching techniques were favored by graduate students. The 
model that was highly valued by all groups of users was "written guides with sample search 
questions." Table 20 summarizes the findings. 

Table 20 

Recommended Instruction Programs by User Status 





Faculty 


Graduate 


Undergrad 


Staff 


Total 


% 




Total % 


Total % 


Total % 


Total % 






Computer assisted 
instruction) 


2 50 


41 39 


46 51 




89 


44.5 


Individual 
instruction 


3 75 


34 33 


43 47 




80 


40 


Subject related 
instruction 




29 28 


25 27 




54 


27 


1 Guide & sample 
questions 


2 50 


38 37 


36 40 


1 100 


77 


38.5 


Scheduled Training 
sessions 


1 25 


35 34 


27 30 




63 


31.5 


Workshop( for 
searching skills) 




39 38 


33 36 




72 


36 


| No instruction is 
| needed 




8 8 


5 5 




13 


6.5 
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The reader might wonder if the instructional program needed by the novice and the 
less experienced user differs from the model demanded by the more experienced searcher. 
According to the results, "CAT and "individual instruction on demand" were popular with 
all users, no matter how much prior experience they had. Compared with the novice and 
the less experienced user, patrons with more CD-ROM search experience showed more 
interest in "written guides with sample search questions" and workshops emphasizing search 
skills. Table 21 reveals the different needs by users with different levels of CD-ROM 
experience. Other suggestions mentioned by the patrons were: handouts near terminals, 
assignment geared for specific results, more terminals always with CDs in them so people 
can teach themselves, teachers informing students of the systems, and classes for different 



people at different levels in computer knowledge. 

Table 21 

Recommended Instruction Programs by User Prior Experience 





0 


1-5 


6-10 


11-15 


15 + 


Total 




Total % 


Total % 


Total % 


Total % 


Total % 




Computer-assisted 
Instruction 


14 56 


30 45 


14 40 


7 41 


24 43 


89 


Individual 
instruction 


11 44 


24 36 


16 46 


6 35 


23 41 


80 


Subject related 
instruction 


6 24 


18 27 


10 29 


6 35 


14 25 


54 


Guide & sample 
questions 


9 36 


25 37 


13 37 


7 41 


23 41 


77 


I Scheduled training 
| sessions 


6 24 


27 40 


14 40 


4 24 


12 21 


63 


Workshop (for 
searching skills) 


6 24 


26 39 


14 40 


4 24 


22 39 


72 


No instruction is 
needed 


2 8 


5 7 


3 9 


1 6 


2 4 


13 
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An open-ended question was at the end of the survey for participants to offer 
comments and suggestions. Comments such as these were common: "easy to use," "very 
helpful," "The system is great." Quite a few commented on the need for more workstations, 
and several suggested making the system accessible from remote computers or through 
Internet. Some complaints about hardware problems were made, and the need for more 
advertising and training were recommended. Representative examples include the following: 

The system is very helpful. 

The product is easy to use. 

The system is great. 

More information about how to retrieve a file of downloaded CD-ROM citations 
utilizing the users' computers is desired. 

Divide the CD-ROM area into two areas, one for experienced users and the other 
for the inexperienced. 

The system is flexible but more time is needed. 

It is very self-explanatory. 

Have viewing screens that do not strain the eyes. 

Hard to find exactly what I need. Have to search through many files. 

More terminals are needed. 

Provide CD-ROM search instruction in the evening hours and announce the schedule 
on GEMINI. 

Extremely time saving. 

"A Day at the Library" training would be great. 

Database is adequate, but supply of journals/books in the Library is very limited. 
I like searching by myself. I also like having a librarian available to help me when 
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I have problems. 

Please make it accessible from home, 
A workshop is a great idea, 

CD-ROMs are much better than the equivalent printed indexes. 
Should be advertised more. 

It would make it easier if the user could know whether or not the library has the 
journals. 

Make it accessible via Internet. 

I feel that it could be helpful if students were aware of the search program for 
research assignments in their classes. 

Use of CD-ROM is much easier than I expected. 

Librarians were more helpful than on-line instructions, especially when students are 
pressed for time. 

Have problems of printing out citations, 

IMPLICATIONS OF THE PROJECT 

The results of the project indicate that patrons made little use of the workshops that 
were provided, and they tended not to use the database search guides available at the 
workstation but rather reqi^sted assistance from the librarian. Seeing that the former is 
good for the basics, and the latter is a desirable self-help tool, ways of encouraging better 
attendance at the instruction workshops should be found. An easy and inexpensive 
improvement would be to move the display rack of the database search guides to a more 
obvious location, so that the patrons would see it immediately and utilize it before starting 
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their searches* 

As to the search strategy employed by the users, graduate students were inclined to 
make better preparation for their searches than undergraduate students did. Nevertheless, 
only a little portion of the patrons used Boolean logic to formulate their search statements. 
This is probably because they did not understand the concepts of Boolean searching. 
Although the precision of search results can be enhanced through the use of the thesaurus, 
index terms available on-line, and the selection of specific terminology, the majority of the 
users failed to do so. Therefore, ways to cultivate users' habits of looking up the 
documentation, as well as develop their concepts of Boolean logic, should be reinforced 
when offering guidance. 

The researcher did not expect that the most experienced or patrons employing more 
advanced search strategy, such as Boolean logic, would not get better search results. Many 
of them got fewer citations than they expected. This interesting finding can be attributed 
to several possibilities. First, even though many users reported using logic operators, they 
may have used natural language instead of indexed terms and thesaurus entries. This could 
have led to false drops. Second, they may have selected search terms which were not 
specific enough, and attempted to perform searches which were not well refined. Third, 
they may have tried to locate information which was outside of the scope of the CD-ROM 
application. And finally, it is uncertain as to whether or not the capability and the limitation 
of the system was fully comprehended. All of these imply that if the patrons gain a better 
understanding of the nature of the databases they use, then properly conducted searches will 
yield more precise results. 
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Several factors influenced the users 5 searches; hardware was one of them. The most 
common complaint about hardware was the printer, so maintenance and providing better 
quality of printers should be seriously considered. Throughout the duration of the survey, 
the researcher observed that even if there were signs to inform the patrons of the databases 
available on network and stand-alone workstations, many users did not notice their 
existence. It is likely that the signs were too small to catch users' attention. This made 
some of the patrons unable to determine which dedicated workstation to choose. Besides, 
these signs listed only the CDs' titles under subject areas without any scope notes. This did 
not reduce the difficulty for inexperienced patrons to select the right databases they needed. 
Users cannot make critical choices about information sources until the nature of indexes is 
understood. Therefore, it is necessary to design informative signs and set them in an 
obvious place. To serve their purpose, s :gns might include the title of the CD-ROM product 
and a brief description of its information -rnverse. Since over 20% of the respondents did 
not know how to retrieve relevant information more effectively by using Boolean logic and 
the thesaurus, the librarians can consider providing training in developing search strategies, 
so that both the ability of patrons to use the CD-ROM equipment and to perform their 
searches can be improved. Training programs that are geared toward problems that users 
may encounter in the course of searching can benefit users, especially novices. 

The CD-ROM producers always claim that they put forth a great effort in making 
the system easy to use. Nevertheless, the results of the survey show that sometimes the 
patron's frustration was caused by the lack of instructions in various user interfaces. Each 
product has its own on-line help, but very often it could not adjust to various situation that 
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users confronted. When an improper command was detected, very likely, the error message 
failed to explain what the error was and prompt the users for missing parameters. For some 
systems, especially UMI products, the more sophisticated the strategy, the longer the search 
takes. Other common product problems identified are lack of flexibility, slow responses 
time, short date range and lack of specific Help Screen. Recognition of these problems 
should be valuable to CD-ROM producers in developing and enhancing their search 
interfaces. Before these drawbacks are improved, the librarian might keep users from 
getting stuck by creating a template, instructing users how to get started, how to get help, 
how to select options from the menu, what the function keys mean, how to look up the on- 
line index or thesaurus, how to truncate, how to print the citations found, and exit from the 
system. Other measures that the product producers and the librarian can take to meet user 
satisfaction are indexing more journals/books, expanding the range cf years covered, 
providing abstracts, furnishing more terminals, indicating the library holdings and loading 
the call numbers of the titles owned by the library into the software. Another crucial 
suggestion for enhancement of the service is allowing the systems to be accessed from 
remote computers or through other avenues. This provides the users with the convenience 
of doing searching at home or in their offices. It will also free the workstations a little bit 
and help decrease the chance of patrons waiting to use the terminals. Additionally, this 
implementation will help reduce the librarians' workload, so higher quality of services can 
be offered. 

CD-ROM technology has been accepted enthusiastically by the users, but because of 
their unfamiliarity with the structure of databases and searching tricks, the users were not 
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able at times to retrieve the reasonable amount of information they needed. The 
instructional programs suggested by the respondents to help them perform a successful 
search were CAI, individual instruction on demand, written guides with sample search 
questions and workshops focusing on the conceptual side of the search, such as choosing the 
proper database, selecting appropriate search terms, using Boolean operators, formulating 
search statements, etc. CAI was the first priority of the majority of users because it can 
offer an interactive means of learning and patrons can get feedback during the learning 
process. Since people who had attended workshops, consulted handouts, looked up the 
thesaurus, and had much CD-ROM experience still relied heavily on the assistance of the 
librarian, there is no doubt that individual instruction on demand was strongly desired. And 
it is clear that the CD-ROM librarian who is available for immediate, problem-specific 
assistance is indispensable. Yet, considering the fact that the library does not have enough 
human resources to offer scheduled, one-on-one instruction, the alternative way might be 
to have an adequate number of end-user searching assistants available at the CD-ROM help 
desk to provide assistance for patrons. The source of assistants can be graduate students, 
from the School of Information Science and Policy, who completed the online database 
searching class or have online bibliographic search experience. One way of making the 
current database guide more beneficial is to add a couple of sample search questions to it. 
This would create the opportunity for the users to put the concepts they have gained into 
practice. And practice makes perfect. A CD-ROM user might conduct a good search 
without using truncation and other search skills; while without an understanding of thesaurus 
use and Boolean search methodology, he/she is not able to use the system more efficiently 
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and perform his/her searches more effectively; without a full comprehension of the nature 
of the database, he/she cannot make critical choices about information sources. That might 
be the reason why a workshop pertinent to developing users' search abilities was highly 
valued. 

CONCLUSION 

CD-ROM technology has changed the way information is accessed and distributed. 
It is important for reference librarians to keep abreast of changes in database content and 
the system software. In the mean time, the librarian needs to prepare well-planned 
instructional programs for teaching end-users to sufficiently utilize the powerful system and 
obtain satisfactory search results. It is true that a user survey can never tell the whole story 
of patrons' reactions to the CD-ROM database service. Still, the responses and comments 
received from the users provided valuable ideas about what they really want and need. 
Their opinions enabled the researcher to address some problems that users encountered, 
too. By revealing the results to the reference department, it is hoped that the librarians 
would benefit from this survey by making the suggested improvements. In addition, the 
librarian might serve as the intermediary to pass along the complaints of the patrons 
surveyed to the CD-ROM producers, so that enhancement can be made when new versions 
of the products are developed. 

CD-ROM databases have become an essential, time-saving tool for research in 
academic and research libraries, still there are gaps to be filled in order to meet user 
satisfaction. Making the products truly user-friendly and competitive with online database 
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search systems, which provide rapid, timely, and cost-effective service, is the major priority 
for the CD-ROM designers to master. And only when the producers, the librarians and the 
patrons cooperate with one another, can the CD-ROM database service be fine tuned to the 
needs of the end-users. 
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APPENDIX A 



Subscribing MEDLINE EXPRESS instead of MEDLINE. 

Provide a way to skip long lists of references. 

Make more specific index terms. 

Index more newspaper , 

Make it easy to switch among search modes. 

Better printers are needed. 

Expand the information stored. 

Have information sheets that give examples of what the computer can do, so we can 
choose what we need. 

A notebook describing F1-F10 for each specialized disk near the terminals would 
help. 

Provide a method to exclude non-English articles. 
Stock more medical journals in the library. 
Include more journals that this library has. 
Expand search procedures. 

Index only journals/books that are available locally. 

List found titles at once on separate menu instead of a number. 

Explain what was being seen. 

Have a librarian whose job is to specially aid those using computers. 
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